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Abstract 

Frames have established themselves as a means to derive redundant, yet stable decom- 
positions of a signal for analysis or transmission, while also promoting sparse expansions. 
However, when the signal dimension is large, the computation of the frame measurements of 
a signal typically requires a large number of additions and multiplications, and this makes a 
frame decomposition intractable in applications with limited computing budget. To address 
this problem, in this paper, we focus on frames in finite-dimensional Hilbert spaces and in- 
troduce sparsity for such frames as a new paradigm. In our terminology, a sparse frame is a 
frame whose elements have a sparse representation in an orthonormal basis, thereby enabling 
low-complexity frame decompositions. To introduce a precise meaning of optimality, we take 
the sum of the numbers of vectors needed of this orthonormal basis when expanding each 
frame vector as sparsity measure. We then analyze the recently introduced algorithm Spectral 
Tetris for construction of unit norm tight frames and prove that the tight frames generated 
by this algorithm are in fact optimally sparse with respect to the standard unit vector basis. 
Finally, we show that even the generalization of Spectral Tetris for the construction of unit 
norm frames associated with a given frame operator produces optimally sparse frames. 

1 Introduction 

Frames are nowadays a standard methodology in applied mathematics, computer science, and 
engineering when redundant, yet stable expansions are required. Examples include sampling 
theory [5D] , data quantization [SJ |3] , quantum measurements [H] , coding [3J , image processing 
[5J [55] , wireless communication (551 [231 130] , time- frequency analysis [HI GH] , speech recognition 
[TJ, and bioimaging [16]; see also [24] [25] for a beautiful survey and further references. The typical 
application exploits the decomposition of a signal x £ R™ into its frame components, which requires 
computation of the frame measurements, i.e., the inner products between the signal x and the 
frame vectors (ifii)fL 1 , say. However, if the dimension n of the ambient space is large and the 
frame vectors have 'many' non-zero entries, the computational complexity of the computation of 
the frame measurements might be high; in fact, for applications with constraints on the available 
computing power and bandwidth for data processing, computing the frame measurements and 
hence the frame decomposition might be intractable. 

In this paper, we focus on frames in finitc-dimcnsional Hilbert spaces and tackle this problem 
by constructing frames which have very few non-zero entries, thereby reducing the number of 
required additions and multiplications when computing frame measurements significantly. This 
viewpoint can be also slightly generalized by assuming that there exists a unitary transformation 
mapping the frame into one having this 'sparsity property'. Sparsity of fusion frames, which were 
introduced in [14] as a mathematical framework for distributed processing thereby going beyond 
frame theory, was already defined in [7] as a concept. However, the paradigm we aim for in this 
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paper differs from the one introduced in [7] for fusion frames when restricting to the case of frames, 
since we here aim for an overall sparsity of the frame. 

Frame constructions have a long history; browsing through the literature, however, it becomes 
evident that all constructions for unit norm tight frames - those frames most advantageous for 
applications - only produce such frames for very special cases such as harmonic frames, see also [1Q[ 
115] . Very recently, a significant advance in the construction of unit norm tight frames was achieved 
through the introduction of the so-called Spectral Tetris algorithm in [12] . For most combinations 
of the number of frame vectors and the dimension of the ambient space, this procedure indeed 
generates a unit norm tight frame. An extension of Spectral Tetris to construct unit norm frames 
with prescribed frame operator if its eigenvalues are greater or equal to two was introduced in [7] 
to allow additional flexibility in the design process. 

In this paper we show that the unit norm frames which this extended Spectral Tetris algorithm 
generates are optimally sparse in the sense of the total number of non-zero entries in the frame 
vectors, provided that Spectral Tetris is performed after ordering the prescribed eigenvalues in an 
appropriate way. We also explicitly determine the exact minimum value of the non-zero entries. 
Along the way, we introduce block decompositions as a novel structural property of unit norm 
frames, which we anticipate to be useful also in other settings. 

1.1 Main Contribution 

Our main contribution is hence two-fold: Firstly, we introduce sparsity of a frame as a novel 
paradigm in frame theory. More precisely, we introduce the notion of a sparse frame as well as 
a sparsity measure for such frames, thereby allowing for optimality results. Secondly, we analyze 
an extended version of Spectral Tetris and prove that this algorithm indeed constructs optimally 
sparse frames if performed after ordering the prescribed eigenvalues blockwisc. Thus, Spectral 
Tetris can serve as an algorithm for computing frames with this desirable property, and our 
results show that it is not possible to derive sparser frames through a different procedure. 

1.2 Impact on Applications 

Frames are nowadays considered a fundamental tool in electrical engineering, and we wish to refer 
to the survey paper [16] as also to the introductory papers (24] [25]. However, the application of 
frames for the analysis of high-dimensional data such a webpages labeled by over a million param- 
eters or databases of images, each image being one data point, typically suffers from the fact that 
frame measurements are computationally not feasible due to constraints such as computing power 
and bandwidth, or even limited space to store the synthesis matrix. With the results presented in 
this paper, we introduce sparsity of frames as a novel paradigm for frame constructions, resulting 
in computationally highly efficient frames. Our results do not only provide a lower bound for 
the maximally achievable sparsity, but with the Spectral Tetris algorithm explicit constructions 
of efficient frames for high-dimensional data analysis are now possible. Certainly, the desire to 
construct tight frames is evident due to the favorable reconstruction properties of such frames. 
But our results go beyond this case, and also enable constructions of optimally sparse non-tight 
frames with prescribed eigenvalues of the frame operator. Let us provide two additional exemplary 
applications illustrating why such frame properties are a natural constraint and which areas our 
results are anticipated to impact. 

Analysis of Streaming Signals. The structure of the frame operator plays a key role in the noise 
rejection ability of the frame. When the frame coefficients arc corrupted by additive white Gaussian 
noise, the mean-squared error (MSE) in reconstructing the signal is minimized by choosing the 
frame to be tight. In the presence of colored noise, however, a tight frame is no longer optimal 
and the frame operator needs to be matched to the noise covariance matrix. In such cases, the 
frame needs to be designed with respect to the eigen-basis of the inverse noise covariance matrix 
and its eigenvalues. This is similar in spirit to the water-filling principle for precoder design in 
wireless communication, where transmit power is distributed across the eigen directions of an 
inverse channel-noise covariance matrix to equalize signal-to-noise-ratio across eigen directions, 
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see [55J H7] . Additional structure on the frame is typically required depending on the application. 
When the signal to be decomposed is a time or space series that cannot be observed or processed 
over long blocks — due to limited memory, aperture size, or computational power — then it is needed 
to have a frame that not only has a prescribed operator but also requires access to the signal 
samples only over a small temporal or spatial window. This motivates construction of frames with 
sparse elements and desired spectra. The constructions we develop in this paper yield to 2-sparse 
frames in any dimension, where the frame coefficients for a signal in an A-dimcnsional space can 
be computed by observing the data stream through a window of only two samples. The frame 
sparsity can be tailored to any arbitrary basis. In particular, when the noise covariance is known, 
the frame can be made sparse with respect to the eigen-basis of the inverse noise covariance matrix. 

Face Recognition. In face recognition, one main objective is to classify faces according to some 
given criterion, for instance, to distinguish male from female faces. The application of PCA (or 
similar algorithms) delivers a basis of cigenfaces. Learning algorithms on some training set of 
faces can then, for each basis element, determine the degree of the significance of its coefficients 
for determining the gender. Customarily, measurements taken to classify faces are assumed to be 
affected by noise; hence frame expansions are desirable. The frame should ideally be designed to 
match the degree of significance (= eigenvalues) of the given basis in the sense that it should be 
more redundant for the computation of the significant coefficients and less for the insignificant 
coefficients. This is precisely the setting we consider in this paper, and for which we analyze and 
construct optimally sparse frames. 

These are just brief samples of applications which will benefit from the results and constructions 
developed in this paper. We anticipate that also various other applications are impacted. 

1.3 Outline 

This paper is organized as follows. In Section [21 we first fix the terminology we require from frame 
theory and then review the extended version of the Spectral Tetris algorithm. A novel sparsity 
measure for a frame will then be introduced in Section [3] together with a notion of optimality. In 
Section |H a structural property of frames suitable for our analysis is first introduced, and finally 
we state and prove our main result Theorem l4.5l We finish with some conclusions and discussions 
in Section [5] 

2 Frame Construction 

We first review the initial as well as the extended version of the Spectral Tetris algorithm from 
[7]. To stand on common ground, we start by fixing our terminology while briefly reviewing the 
basic definitions and notations related to frames. 

2.1 Frames 

A sequence <J> = (</?i)£Li in R™ is called a frame for R™, if it is a - typically, but not necessarily 
linearly dependent - spanning set. This definition is equivalent to asking for the existence of 
constants < A < B < oo such that 

N 

A\\x\\ 2 < \(x,<Pi)\ 2 < B\\x\\ 2 for all x g R". 

i=l 

When A is chosen as the largest possible value and B as the smallest for these inequalities to hold, 
then we call them the (optimal) frame bounds. If A and B can be chosen as A — B, then the frame 
<f> is called A-tight, and if A = B = 1 is possible, $ is a Parseval frame. $ is called equal-norm, if 
there exists some c > such that \\ifi\\ = c for all i = 1, . . . , N, and it is unit-norm if c = 1. 

Frames allow the analysis of data by studying the associated frame coefficients {{x,ipi))f =1 , 
where the operator T defined by T : R™ — > ^({l, 2, . . . , N}), x H> ((x, ifii))fLi is called the analysis 
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operator. The adjoint T* of the analysis operator is typically referred to as the synthesis operator 
and satisfies T*((cj)^. 1 ) = $3»=i ^fi- Later, the synthesis operator will play an essential role, and 
we will write it in the matrix form [cpi| . .. |(/5jv] with the frame vectors as columns. In the sequel 
we refer to this matrix as the synthesis matrix. The main operator associated with a frame, which 
provides a stable reconstruction process, is the frame operator 

N 

S = T*T : R n -> R n , x ^ J^(z, tpi)ipi, 

i=l 

a positive, self-adjoint, invertible operator on R™. In the case of an A- tight frame, we have 
S = A ■ IdjRn , and in case of a Parseval frame, S = Idg™ . In general, S allows for the reconstruction 
of a signal x € R™ through the reconstruction formula 

N 

x = ^{x,S~ 1 ipi)(p i . (1) 

i=l 

Redundancy is obviously the crucial property of a frame ensuring resilience to noise and erasures 
while simultaneously enabling us to choose the expansion coefficients appropriately. The particular 
choice of coefficients displayed in (TIJ is the smallest in £2 norm [T7] . hence it contains the least 
energy. Recently, a different view point has received rapidly increasing attention, namely to choose 
the coefficient sequence to be sparse in the sense of having only few non-zero entries, thereby 
allowing data compression while preserving perfect recoverability (see, e.g., [5], and the references 
therein). In this context, for later use, we will denote the support of a vector x £ R™, i.e., the set 
of indices of the non-zero entries, by suppx. 

Finally, we should mention that, customarily, redundancy of a frame (</5j)t=i f° r ^™ was mea- 
sured by ^, i.e., the number of frame vectors divided by the dimension of the ambient space. Since 
this measure is exceptionally crude and not sensitive to local behavior of the frame vectors, the 
notions of upper and lower redundancy have been suggested in [4] as a finer redundancy measure. 

2.2 The Spectral Tetris Algorithm 

Spectral Tetris was first introduced in |12j as an algorithm to generate unit norm tight frames for 
any number of frame vectors N, say, and for any ambient dimension n provided that — > 2. This 
algorithm is indeed significant for frame constructions, since it is the first systematic construction 
of unit norm tight frames. Before, only a number of very special classes of unit norm tight frames 
such as harmonic frames have been known. 

An extension to the construction of unit norm frames having a desired frame operator associ- 
ated with eigenvalues Ai, . . . , A n > 2 satisfying $^3=1 \j = N was then introduced and analyzed 
in [7] - in fact, an even more general algorithm for the construction of fusion frames was stated 
therein. The frame- version of this algorithm is what we intend to analyze in this paper. Figure [T] 
states the steps of this version of the algorithm, which we coin Spectral Tetris for Frames; in short, 
STF. We wish to remark that the original form of the algorithm in [12] requires the sequence of 
eigenvalues to be in decreasing order, i.e. X\ > . . . > X n . This assumption, however, was made 
only for classification reasons, and it is easily seen that it can be dropped. Since in the sequel, 
we will consider carefully chosen, presumably non-decreasing, sequences of eigenvalues, the gained 
freedom is essential for our analysis. 

Before we continue, let us give an example to provide a more intuitive feeling of how STF 
works and to introduce a cursor notation which will be utilized in later proofs. 

Example 2.1. We aim to construct a 10 element unit norm frame in R 4 having the eigenvalues 
Ai = A2 = A3 = I and A4 = 2. STF will provide such a frame by generating a 4 x 10 synthesis 
matrix with the following properties: The columns of the matrix have norm 1 (guaranteeing that 
the frame has unit norm vectors ) and the rows of the matrix are orthogonal and square sum to the 
desired eigenvalues ( guaranteeing that the frame operator is diagonal with the desired eigenvalues 
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on its diagonal). In the example we consider each of the first 3 rows of the to-be-generated synthesis 
matrix have to square sum to | and the last row to square sum to 2. The algorithm now starts 
with a 4 x 10 matrix of unknown entries and lets a cursor move forward along columns and rows 
assigning values to certain entries. The remaining entries are set to zero in the end. When the 
cursor is in position (i,j), we update the variable Xi to be the difference between the eigenvalue 
assigned to row i and the square sum of the entries already assigned to row i, in order to keep 
track of how much weight still has to be assigned to row i to make it square sum to the desired 
eigenvalue. In general, one of the three cases occurs in each step: 

Case 1: If Xi > 1, then the current entry (i,j) is set to one, we update A,; := A; — 1, and the 
cursor is moved to the right, i.e., (i,j) ■= (i + This is, for example, the case when the 

cursor is in position (1, 1). At this point we have Ai = | and we update to Ai = | — 1 = |. The 
matrix changes as follows, where we denote the unknown matrix entries by ■ and the position of 
the cursor by ©: 







1 



Case 2: If < A,- < 1, then the entries (i + + 1), and (i + l, j + 1) are set 

according to lines 5) and 6) of STF, we update A^+i := A; + i + A; — 2, and the cursor is moved to 
:= (i + 2,j + 1). This is, for example, the case when the cursor is in position (1, 3). At this 
point we have Ai = | and update A2 = I to A2 = § — (2 — |) = |. The matrix changes as follows: 



1 1 



1 1 







This case is the crucial step of the algorithm. Note that the 2x2 block we inserted has the properties 
that its rows are orthogonal and its columns square sum to 1, which are properties desired for the 
synthesis operator. 

Case 3: If Xi = 1, then the entry (i,j) is set to one, and the cursor is moved to the right below 
(i,j) := (i + l,j + 1). This is, for example, the case when the cursor is (3,8). At this point we 
have A3 = 1 and the matrix changes as follows: 



1 1 







1 1 







After performing all steps of the algorithm, the final synthesis matrix constructed by STF has 
the form 



1 1 










1 







i 



1 
1 1 
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3 New Paradigm for Frame Constructions: Sparsity 



3.1 Classical Sparsity 

Over the past few years, sparsity has become a key concept in various areas of applied mathematics, 
computer science, and electrical engineering. Sparse signal processing methodologies explore the 
fundamental fact that many types of signals can be represented by only a few non-zero coefficients 
when choosing a suitable basis or, more generally, a frame. A signal represcntable by only k, say, 
basis or frame elements is called k- sparse. If signals possess such a sparse representation, they 
can in general be recovered from few measurements using l\ minimization techniques (see, e.g., 
[6j El [19] and the references therein). 

3.2 Sparse Frames 

In this paper, however, we pose a different question concerning sparsity, viewing sparsity from a 
very different standpoint. Typically, data processing applications face low on-board computing 
power and/or a small bandwidth budget. When the signal dimension is large, the decomposition of 
the signal into its frame measurements requires a large number of additions and multiplications, 
which may be infeasible for on-board data processing. Also the space required for storing the 
synthesis matrix of the frame might be huge. It would hence be a significant improvement, if 
each frame vector would contain very few non-zero entries, hence - phrasing it differently - be 
sparse in the standard unit vector basis, which ensures low-complexity processing. Since we are 
interested in the performance of the whole frame, the total number of non-zero entries in the frame 
vectors seems to be a suitable sparsity measure. This viewpoint can also be slightly generalized 
by assuming that there exists a unitary transformation mapping the frame into one having this 
'sparsity' property. 

3.3 Sparseness Measure 

Taking these considerations into account, we are led to proclaim the following definition for a 
sparse frame: 

Definition 3.1. Let (ej)™ =1 be an orthonormal basis for R n . Then a frame (<y2i)f=i for R" is 
called fc-sparse with respect to (ej)™ =1 , if, for each i £ {1, ...,-AT}, there exists Ji C 
such that 

tpi £ span{ej : j £ Ji] 

and 

n 

J2\M = k- (2) 

1=1 

The attentive reader will have realized that this definition differs from the definition stated in 
[7] for fusion frames (see [M]) when restricting to the special case of frames. The exact relation is 
the following: Let (e 3 )" =1 be an orthonormal basis for R™, let {y>i)f =1 be a frame for R n , and, for 
each i £ {1, . . . , N}, let Ji C {1, . . . , n} such that (pi £ spanje^ : j £ Ji}. Then, in the sense of [7], 
the frame is max{| Ji\ : i = 1,. . . ,n}-sparse, whereas in our Definition 13. 1[ the frame is X)"=i W 
sparse. Thus our definition encodes the true overall sparsity which is the sparsity required for 
frame processing in contrast to the more local version of [7]. 

One can certainly imagine other sparsity measures dependent on the requirements and con- 
straints of the application at hand. Instead of pj), a weighted version could be considered with 
the weights chosen depending on the computational constraints of the application. Also, ([2]) could 
be regarded as the t\ norm of the sequence {\Ji\ : i — 1, . . . and a different viewpoint might 
lead us to considering a different norm instead - as it was done in [7] for the £ oo norm. 
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3.4 Notion of Optimality 

We now have the necessary machinery at hand to introduce a notion of an optimally sparse frame. 
Optimality will typically - as also in this paper - be considered within a particular class of frames, 
for instance, in the class of unit norm tight frames. 

Definition 3.2. Let J 7 be a class of frames for R™, let ((fii)^L 1 G T, and let (ej)" =1 be an 
orthonormal basis for R™. Then (fi) 1 ^ =1 is called optimally sparse in T with respect to (ej)" =1 , if 
{fi)f=i * s ki-sparse with respect to (e,')j =1 and there does not exist a frame (ipi)iLi G J- which is 
ki-sparse with respect to (ej)™ =1 with hi < k\. 

Wc wish to emphasize the strong dependence of sparsity on the chosen basis. Also an optimally 
sparse frame is in general not uniquely determined; we present an example for this observation in 
Subsection g^l 

4 An Optimality Result for Sparse Frames 

We now seek a construction for an optimally sparse unit norm frame with prescribed properties. 
As already elaborated upon before, the condition we impose is having a given frame operator, 
which, in particular, also includes operators with equal eigenvalues corresponding to tight frames. 
This frame operator will in the following be always determined by its eigenvalues. Hence we are 
interested in optimal sparsity within the following class: 

Let n,N > and let the real values Ai, . . . , X n > 2 satisfy J2]=i = N. Then the class of 
unit norm frames (<Pi)fLi m whose frame operator has eigenvalues Ai, . . . , A„ will be denoted 
by 

It is important to mention that by writing {Ai}™ =1 , we wish to indicate that the ordering docs 
not play a role here, however, multiplicities are counted. The just defined class J r (A r , {Ai}" =1 ) is 
non-empty by application of the STF. In fact, it can be shown using methods introduced in [13j 
and [TU], that it is an infinite set for any n, N > and real values Ai, . . . , A„ > 2. 

It might be beneficial for the reader to mention at this point that we will discuss the analysis 
presented in Subsections 14. II to I4.3l in the important special case of tight frames in Subsection 14.41 
for illustrative purposes. 

4.1 Novel Structural Property of Synthesis Matrices 

Aiming for determining the maximally achievable sparsity for a class J-(N, {Ai}™ =1 ), wc first need 
to introduce a particular measure associated with the set of eigenvalues {A,}" =1 . This measure 
indicates the maximal number of partial sums which are an integer; here one maximizes over all 
rcordcrings of the eigenvalues. Before stating the precise definition, let us provide some intuition 
why the maximally achievable sparsity is dependent on partial integer valued sums. Whenever 
the partial sum of the first /, say, eigenvalues Ai, . . . , A; is an integer, the cursor - recall that 
the concept of a cursor was introduced in Example 12.11 - in row I will be in Case 3 of the cases 
discussed in Example 12.11 Opposed to the setting of 4 new non-zeros entries in Case 2, in this 
case only one new non-zero entry will be defined. Roughly speaking, this will allow us to reduce 
the analysis to the blocks between two such integer partial sums. 

The precise definition of the measure on a set of eigenvalues wc require is now as follows. 

Definition 4.1. A finite sequence of real values Ai,...,A„ is ordered blockwise, if for any 
permutation tt of {1, . . . , n} the set of partial sums {Y2j=i ■ s = 1, •••,"■} contains at least 
as many integers as the set '■ s = l,...,n}. The maximal block number of a fi- 

nite sequence of real values Ai,...,A n , denoted by /Lt(Ai, . . . , A n )> is the number of integers in 
{^2j =1 Ao-y) : s = 1, . . . , n}, where a is a permutation of {1, ... ,n} such that > ■ ■ ■ j ^a(n) * s 
ordered blockwise. 
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Surprisingly, the notion of maximal block number can illuminat ingly be transferred to a particu- 
lar decomposition property of the synthesis matrix of a frame. Let us first define the decomposition 
property we are interested in: 

Definition 4.2. Let n, N > 0, and let {fi)f—i be a frame for M. n . Then we say that the synthesis 
matrix of (</?i)i^=i has block decomposition of order m, if there exists a partition {1, . . . , N} = 
II U . . . U I m such that, for any k\ S Ii t and ki G I l2 with i\ ^ ii, we have supp (fk ± f~l supp ipk 2 = 
and m is maximal. 

The following result now connects the maximal block number of the sequence of eigenvalues of 
a frame operator with the block decomposition order of an associated frame. 

Proposition 4.3. Let n, N > and let the real values X%, . . . , X n > 2 satisfy Y^=i -\? = N. Then 
the synthesis matrix of any frame in the class J-(N, {Ai}" =1 ) has block decomposition of order at 
most n(Xi, . . . , A n ). 

Proof. Suppose {^Pi)fLi S J~(N, {Ai}™ =1 ) has block decomposition of order v and let {1, . . . , N} = 
Ii U . . . U L v be a corresponding partition. For j = 1 , . . . , v, let Sj be the common support set 
of the vectors (<Pi)iei-, i.e., k € Sj if and only if k G suppy^ for some i £ Ij. Now let r^ denote 
the fc-th row of the synthesis matrix of (</2i)i^i- Then Si, . . . , S v is a partition of {1, . . . , n) and, 
for every j = 1, . . . , v we have by the fact that {(Pi)f =1 consists of unit norm vectors and by our 
choice of Ij and Sj that 

i^i = Eii^ii 2 = Ei^ii 2 =E a ^ ( 3 ) 

kei, fceSj keSj 

The last equality holds since we, after permutation of the columns, can write the synthesis matrix 
°f (fi)iLi as T* = [T*, . . . ,T*], where T* has zero entries except on the rows indexed by Ij and 
the columns indexed by Sj, for j = \,...,v. The frame operator T*T — Ylj=i TjTj is block 
diagonal with blocks T*Tj, hence its eigenvalues are exactly the union of those of each matrix 
T*Tj. But J2kei 3 ll^fcll 2 = J2keSj \\ r k\\ 2 equals the square of the Hilbert-Schmidt norm of T* and 
therefore the sum of the eigenvalues of T*Tj. This shows the last equality of ((3]). Since ((3]) holds 
for all j = 1, . . . , v , we conclude that the maximal block number of Ai, . . . , A n is at least v. Thus 
the synthesis matrix of the arbitrarily chosen frame (yi)fli m the class -^(iV, {Ai}™ =1 ) has block 
decomposition of order at most \x(Xi, ■ ■ ■ , A n ). □ 

4.2 Maximally Achievable Sparsity 

Having introduced the required new notions, we are now in a position to state the exact value 
for the maximally achievable sparsity for a class T(N, {Xi}f =1 ). It is not initially clear that this 
optimal sparsity can always be attained. With Theorem 14.51 we will prove that this is indeed the 
case; in fact, Theorem 14.51 also provides an explicit construction of those frames. 

Theorem 4.4. Let n, N > 0, and let the real values Xi, . . . , X n > 2 satisfy 5ZJ=i Aj = N. Then 
any frame in F(N, {A,}™ =1 ) has sparsity at least 

N + 2(n-fi(Xi,...,X n )) 

with respect to any orthonormal basis. 

Proof. We first study the case that £t(Ai, . . . , A„) = 1. For this, let T* denote the synthesis matrix 
of a frame in T(N, {Ai}™ =1 ) with respect to a fixed orthonormal basis. For the sake of brevity, in 
the sequel we will use the phrase that two rows of T* have overlap of size k, if the intersection of 
their supports is a set of size k. Note that, since the rows of T* are orthogonal, it is not possible 
that two rows of T* have overlap 1. 

Fix now an arbitrary row r\ of T*. Since, by Proposition 14.31 T* has block decomposition of 
order 1, there exists a row r-i whose overlap with ri is of size > 2. Similarly, there has to exist a 
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row different from n and r-i which has overlap of size > 2 with r\ or r<i. Iterating this procedure 
will provide an order r\,T2, ■ ■ ■ r n such that, for each row rj, there exists some k < j such that rj 
has overlap of size > 2 with rfc. Since all columns in T* are unit norm, for each column c, there 
exists a minimal j for which the entry c rj is non-zero. This yields N non-zero entries in T*. In 
addition, each row T2 through r n has at least 2 non-zero entries coming from the overlap, which 
are different from the just accounted for N entries, since these entries cannot be the non-zero 
entries of minimal index of a column due to the overlap with a previous row. This sums up to 
a total of at least 2(n — 1) non-zero coefficients. Consequently, the synthesis matrix has at least 
N + 2(n — 1) non-zero entries, as desired. 

Finally, suppose fi := fi(\i, . ■ . , A„) > 1. By Proposition 14. 31 T* has block decomposition 
of order at most fi. Performing the same construction as above, there exist at most \i rows rj 
(including the first one) which do not have overlap with a row rjt, k < j. Thus the synthesis 
matrix T* must at least contain N + 2(n — fi) non-zero entries. □ 

It should be mentioned that an optimally sparse frame from J-(N, {Ai}" =1 ) is in general not 
uniquely determined. Various examples can be constructed along the following line: For simplicity, 
we choose n = 4 and N = 9 and construct a tight frame, i.e., Ai = . . . = A4 = |. Then, by Theorem 
14.41 the maximally achievable sparsity is 9 + 2(4— 1) = 15. The following matrices are synthesis 
matrices with respect to the standard unit vector basis of two different frames in J-(9, {|}f = i)j 
the first in fact being generated by Spectral Tetris: 

1 1 y§ 4/i 

\/i \/? \/4 00 

\fi ~\fl \fi >Jl 

000 «y§ -a/§ 1 

and 

ly^^fo 000 

n /3 13 13 13 13 13 n n 

V 8 V 8 V 8 V 8 V 8 V 8 " u 

v/| -a/§ 10 

00 y§ -y^f 1 

4.3 Main Result 

Having set the benchmark, we now prove that frames constructed by Spectral Tetris in fact achieve 
the optimal sparsity rate. For this, we would like to remind the reader that the frame constructed 
by Spectral Tetris (see Figure[T]) was denoted by STF(iV; Ai, . . . , A„). 

Theorem 4.5. Let n,N > 0, and let the real values Ai,...,A n > 2 be ordered blockwise and 
satisfy 2j=i = N. Then the frame STF(N; Ai, . . . , A n ) is optimally sparse in J-(N, {A,}" =1 ) 
with respect to the standard unit vector basis. That is, this frame is N+2(n—fx(Xi, . . . , X n ))-sparse 
with respect to the standard unit vector basis. 

Proof. Let (}fi)f = i be the frame STF(7V; Ai, . . . , A„). We will first show that its synthesis matrix 
has block decomposition of order ijl := /z(Ai, . . . , A„). For this, let ko = 0, and let ki, . . . , £ N 
be chosen such that TOj := Ylj=i ^ s an integer for every i = 1, . . . , /i. Moreover, let m = 0. 
Further, note that fc M = n and m M = N, since 5ZJ=i ^ s an integer by hypothesis. The steps 
of Spectral Tetris (STF) for computing STF(mi; Ai, . . . , X kl ) and STF(7V; Ai, . . . , A„) coincide 
until the cursor index in computing STF(iV; Ai, . . . , A„) reaches (ki,nii). Therefore, the first 
ki entries of the first toi vectors of both constructions coincide. Continuing the computation of 
STF(7V; Ai, . . . , A„) will set the remaining entries of the first mi vectors and also the first k± entries 
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of the remaining vectors to zero. Thus, any of the first k\ vectors has disjoint support from any 
of the vectors constructed later on. Repeating this argument for &2 until k^, we obtain that the 
synthesis matrix has a block decomposition of order fi; the corresponding partition of the frame 
vectors being 



u 



Wmt-i+l, • ■ • 7 fm,}- 
i=l 

To compute the number of non-zero entries in the synthesis matrix generated by Spectral Tetris, 
we let i 6 {1, . . . be arbitrarily fixed and compute the number of non-zero entries of the vectors 
VWj-i+i! • ■ • j fmi- Spectral Tetris ensures that each of the rows + 1 up to ki — 1 intersects the 
support of the subsequent row on a set of size 2, since in these rows STF will always proceed as 
in case 2 of the three cases in the spectral tetris example above. Thus, there exist 2(fc^ — fci_i — 1) 
frame vectors with two non-zero entries. The remaining (to.; — m^-i) — 2(ki — — 1) frame 
vectors will have only one entry, yielding a total number of (m; — to;_i) + 2(fcj — ki-i — 1) non-zero 
entries in the vectors <^ TOil +i, . . . , f> mi . 

Summarizing, the total number of non-zero entries in the frame vectors of (ifii)^Li i s 



^(m, - mi_i) + 2(ki - k^ x - 1) = I ^(?^ - m^i) + 2 /c M . - 1 = N + 2(n - /i) 




which is by Theorem 14.41 the maximally achievable sparsity. □ 

The reader will have realized that Spectral Tetris generates frames which are 'only' optimally 
sparse with respect to the standard unit vector basis. This seems at first sight like a drawback. 
However, if sparsity with respect to a different orthonormal basis is required, Spectral Tetris can 
easily be modified to accommodate this request by using vectors of this orthonormal basis instead 
of the standard unit vector basis when filling in the frame vectors in Steps 5, 6 and 11 in STF (cf. 
Figure [1} . It is a straightforward exercise to show that this modified Spectral Tetris algorithm 
then generates a frame which is optimally sparse with respect to this new orthonormal basis. 



4.4 Special Case: Constructing Optimally Sparse Tight Frames 

In the special case of equal eigenvalues, i.e., of tight frames, with N elements in R™, all eigenvalues 
need to equal for the equality 5Z™ =1 A? = N to be satisfied. The maximal block number can be 
easily computed to be gcd(iV, n). Theorem 14.51 then takes the following form: 

Corollary 4.6. For n, N > 0, the frame STF(N; . . . , f ) is optimally sparse in J"(iV, {f }f =1 ) 
with respect to the standard unit vector basis. That is, this frame is N + 2(n — gcd(N,n))- sparse 
with respect to the standard unit vector basis. 



5 Conclusions and Discussion 

In this paper we considered the design of frames which enable efficient computations of the asso- 
ciated frame measurements. This led to the introduction of the notion of a sparse frame as well as 
a sparsity measure for such frames, thereby introducing optimal sparsity as a new paradigm into 
the construction of frames. We then analyzed an extended version of Spectral Tetris for frames 
and proved that the frames constructed by this algorithm are indeed optimally sparse. This shows 
that Spectral Tetris can serve as an algorithm for computing frames with this desirable property, 
and our results prove that it is not possible to derive sparser frames through a different procedure. 

We would finally like to point out that the analysis in this paper leads to several intriguing 
open problems for future research; a few examples are stated in the sequel. 

• Eigenvalues also smaller than 2. It is still an open problem whether and how Spectral Tetris 
extends to sets of eigenvalues, if some eigenvalues are smaller than 2. The extension of the 
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Spectral Tctris algorithm by inserting larger DFT matrices than the previously exploited 
2 x 2-matrices, allowed for some partial results (see [TT]). However, from the results in 
[TTj it can be deduced that this procedure does not always lead to optimally sparse frames 
even in the case when all eigenvalues are equal, i.e., the tight frame case. Hence, extensive 
research will be necessary to introduce an appropriate - in the sense of optimal sparsity - 
extension of Spectral Tctris. 

• Extension to other classes of frames. Depending on the application, other desiderata might 
be requested from a frame such as, for instance, equi-angularity. For such a class of frames, 
the question of an optimally sparse frame can and should similarly be posed. 

• Relative sparsity /Compressibility. Taking numerical considerations and perturbations into 
account, it will be necessary to extend the notion of sparsity to relative sparsity/comprcssibility 
for frames and analyze optimality for such. 
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STF: Spectral Tetris for Frames 

Parameters: 

• Dimension n G N. 

• Number of frame elements N € N. 

• Sequence of eigenvalues Ai, . . . , A n > 2 satisfying J2]=i <\j = N. 

Algorithm: 

1) Set i := 1. 

2) For j = 1, . . . ,n do 



3) Repeat 

4) If < 1 then 

5) <pi := y^f • ej + y 1 - 4* • ej +x . 

6) := -y/^ • e,- - ■ e J+ i . 

7) i:=i + 2. 

8) A 3+ i := Aj+i - (2 — Aj). 

9) Aj := 0. 

10) else 

11) <^ := e 3 -. 

12) i:=i + l. 

13) A, := A 3 - 1. 

14) end. 

15) until Aj = 0. 



16) end. 
Output: 

• Frame STF(7V; Ai, . . . , A n ) := {^}?=i- 

Figure 1: The Spectral Tetris algorithm for constructing an iV-element unit norm frame 
STF(iV; Ai, . . . , An) for M. n with an associated frame operator having eigenvalues Ai, . . . , A„. 
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